Evaluating Robust Design Methods Using a Model of Interactions in Complex Systems

Uncertainty, robustness, and flexibility are three of the issues widely viewed as central to the field of complex engineered systems. A key challenge is to make the consideration of these issues more quantitative and theoretically sound. This paper presents a detailed technical analysis of one question inter-relating all three issues. How can we quantify the benefits of flexibility in the design processes used to seek robustness to uncertainty? To address this question, an approach is presented for evaluating robust design methods via probabilistic simulation. Different robust design methods are repeated on a large number of systems and the results are analyzed statistically. A key to the approach is appropriate modeling of interactions among the variables in complex engineering systems. The weak heredity model is used to generate simulated systems with the properties of effect sparsity, hierarchy, and inheritance. A case study is presented in which different combinations of factorial designs and adaptive plans are used in robust design experiments. It is shown that a combination of an adaptive inner array (a flexible plan) with a fractional factorial outer array (an efficient but inflexible plan) provides the best results. Thus, the case study provides specific operational advice for deploying flexibility in one particular systems engineering task. Because of this bit of progress, the new evaluation technique is proposed one example of rigorous quantitative treatment of foundational issues in complex engineering systems made possible by combining advances in computing power and statistical theory.